Multiple digital assistant coordination in vehicular environments

ABSTRACT

The present disclosure is generally related to a data processing system to selectively invoke applications for execution. A data processing system can receive an input audio signal and can parse the input audio signal to identify a command. The data processing system can identify a first functionality of a first digital assistant application hosted on the data processing system in the vehicle and a second functionality of a second digital assistant application accessible via a client device. The data processing system can determine that one of the first functionality or the second functionality supports the command. The data processing system can select one of the first digital assistant application or the second digital assistant application based on the determination. The data processing system invoke one of the first digital assistant application or the second digital assistant application based on the selection.

BACKGROUND

Excessive network transmissions, packet-based or otherwise, of networktraffic data between computing devices can prevent a computing devicefrom properly processing the network traffic data, completing anoperation related to the network traffic data, or responding timely tothe network traffic data. The excessive network transmissions of networktraffic data can also complicate data routing or degrade the quality ofthe response when the responding computing device is at or above itsprocessing capacity, which may result in inefficient bandwidthutilization. A portion of the excessive network transmissions caninclude transmissions for requests that are not valid requests.

SUMMARY

According to an aspect of the disclosure, a system to selectively invokeapplications for execution can include a data processing system of avehicle. A natural language processor component executed by the dataprocessing system can receive, via an interface, an input audio signal.The natural language processor component can parse the input audiosignal to identify a request and a command corresponding to the request.A configuration tracker executed by the data processing system canidentify a first functionality of a first digital assistant applicationhosted on the data processing system in the vehicle and a secondfunctionality of a second digital assistant application accessible via aclient device communicatively coupled to the data processing system. Anapplication selector executed by the data processing system candetermine that one of the first functionality or the secondfunctionality supports the command corresponding to the request. Theapplication selector can select one of the first digital assistantapplication hosted on the data processing system or the second digitalassistant application accessible via the client device based on thedetermination that one of the first functionality or the secondfunctionality supports the command. The application selector can invokeone of the first digital assistant application or the second digitalassistant application, selected based on the determination that one ofthe first functionality or the second functionality supports thecommand.

According to an aspect of the disclosure, a method to validate vehicularfunctions can include a natural language processor component executed bya data processing system of a vehicle receiving, via an interface, aninput audio signal. The method can include the natural languageprocessor component parsing the input audio signal to identify a requestand a command corresponding to the request. The method can include aconfiguration tracker executed by the data processing system identifyinga first functionality of a first digital assistant application hosted onthe data processing system in the vehicle and a second functionality ofa second digital assistant application accessible via a client devicecommunicatively coupled to the data processing system. The method caninclude an application selector executed by the data processing system,determining that one of the first functionality or the secondfunctionality supports the command corresponding to the request. Themethod can include the application selector selecting one of the firstdigital assistant application hosted on the data processing system orthe second digital assistant application accessible via the clientdevice based on the determination that one of the first functionality orthe second functionality supports the command. The method can includethe application selector invoking one of the first digital assistantapplication or the second digital assistant application, selected basedon the determination that one of the first functionality or the secondfunctionality supports the command.

These and other aspects and implementations are discussed in detailbelow. The foregoing information and the following detailed descriptioninclude illustrative examples of various aspects and implementations andprovide an overview or framework for understanding the nature andcharacter of the claimed aspects and implementations. The drawingsprovide illustration and a further understanding of the various aspectsand implementations, and are incorporated in and constitute a part ofthis specification.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings are not intended to be drawn to scale. Likereference numbers and designations in the various drawings indicate likeelements. For purposes of clarity, not every component may be labeled inevery drawing. In the drawings:

FIG. 1 illustrates an example system to selectively invoke applicationsfor execution.

FIG. 2 illustrates a top view of the vehicle and illustrates theinterior cabin of the vehicle illustrated in FIG. 1 .

FIG. 3 illustrates a method to selectively invoke applications forexecution using the example system illustrated in FIG. 1 .

FIG. 4 is a block diagram of an example computer system.

DETAILED DESCRIPTION

Following below are more detailed descriptions of various conceptsrelated to, and implementations of, methods, apparatuses, and systems toselectively invoke applications for execution in vehicular environments.The vehicular environment can include the vehicle and its surroundings,including a radius around the vehicle (e.g. 100 feet) as well as devicesthat are connected to the vehicle via wired or wireless electroniccommunication systems. The various concepts introduced above anddiscussed in greater detail below may be implemented in any of numerousways.

The present disclosure is generally directed to a data processing systemto selectively invoke applications for execution in vehicularenvironments. The systems and methods described herein can include adata processing system in a vehicle that can receive an audio inputquery, which can also be referred to herein as an audio input signal.From the audio input query, the data processing system can identify arequest. The request can correspond to a command to perform one ofvarious functions, such as a vehicular function (e.g., to open a windowor to activate a windshield wiper) or a non-vehicular function (e.g., tomake a phone call or to access a calendar). Digital assistantapplications can carry, obtain, respond to, or process the commandextracted from the audio input query. The digital assistant applicationcan be a program or script executed on the data processing system, thevehicle, and the client device (e.g., a smartphone, tablet, laptop,desktop, etc.) interfacing with the data processing system via thecloud. The digital assistant application can receive audio inputqueries, process the requests associated with such queries using naturallanguage processing algorithms, and then present an audio response in aconversational manner.

There can be multiple digital assistant applications available in thevehicular environment, each with various functionalities to carry outthe requested command. Certain commands may be performed by a primarydigital assistant application hosted on the data processing system inthe vehicle, while other commands can be performed by a secondarydigital assistant application accessible via a client device (e.g.,smartphone or tablet). The client device and the data processing systemcan have established a communication link with each other. Even withcommands that may be carried out either on the data processing system orthrough the client device, some commands can be executed faster or moreefficiently at one digital assistant application versus the otherdigital assistant application because such commands may be performed vianetwork access. With vehicular environments, however, there may belimited connectivity to the network. The data processing system on thevehicle itself may not be connected to the network. In addition, theclient device may move in and out of range of wireless networks. Assuch, the functionalities of either the primary or secondary digitalassistant application may vary.

To carry out the identified command, the data processing system canidentify functionalities of the primary digital assistant applicationhosted in the vehicle and functionalities of the secondary digitalassistant application accessible via the client device. With thefunctionalities of both applications identified, the data processingsystem can determine which digital assistant application supports thecommand as requested in the audio input query. Based on thedetermination, the data processing system can select which of theprimary or secondary digital assistant application to use in carryingout the requested command. Once selected, the data processing system caninvoke the corresponding digital assistant application. When the primarydigital assistant application is selected, the data processing systemcan execute the requested command via the primary digital assistantapplication. Conversely, when the secondary digital assistantapplication is selected, the data processing system can send a commandto the client device via the communication link to carry out therequested command using the secondary digital assistant application.

The present solution can improve efficiency and effectiveness ofauditory data packet processing by selecting the appropriate digitalassistant application to perform the command of the request contained inthe auditory data packet. As the functionalities may vary across themultiple digital assistant applications, the present solution also canreduce network bandwidth and computing resource consumption by selectingthe digital assistant application that can perform the command moreefficiently than the other applications. From a human-computerinteraction (HCI) perspective, the seamless transition in selectivelyinvoking one of multiple digital assistant applications may improve theutility of all the digital assistant applications available in thevehicular environment.

FIG. 1 illustrates an example system 100 to selectively invokeapplications for execution in vehicular environments. The system 100 caninclude at least one data processing system 102, one or more clientdevices 132, and one or more vehicles 144. The one or more clientdevices 132 can be communicatively coupled to the one or more vehicles144, and vice-versa. The at least one data processing system 102, one ormore client devices 132, and one or more vehicles 144 can becommunicatively coupled to one another via the network 146.

The data processing system 102 can include an interface 104. The dataprocessing system 102 can include a natural language processor (NLP)component 106 to parse audio-based inputs. The data processing system102 can include an interface management component 108 to detect andmanage the interfaces of other devices in the system 100. The dataprocessing system 102 can include an audio signal generator component110 to generate audio-based signals. The data processing system 102 caninclude a direct action application programming interface (API) 112. Thedata processing system 102 can include a response selector component 114to select responses to audio-based input signals. The NLP component 106,the interface management component 108, the audio signal generatorcomponent 110, the data repository 122, the direct action API 112, andthe response selector component 114 may be part of a digital assistantapplication 142. The data processing system 102 can include a datarepository 122. The data repository 122 can store parameters 124,policies 126, response data 128, and templates 130. The data processingsystem 102 can also include a configuration tracker component 116, anapplication selector component 118, and a performance evaluatorcomponent 120, among others.

The functionalities of the data processing system 102, such as thedigital assistant application 142, can be included or otherwise beaccessible from the one or more client devices 132 and the one or morevehicles 144. The functionalities of the data processing system 102 maycorrespond to the functionalities or interface with the digitalassistant application 142 executing on the client devices 132 or thevehicles 144. The client devices 132 can each include and execute aseparate instance of the one or more components of the digital assistantapplication 142. The client devices 132 can otherwise have access to thefunctionalities of the components of the digital assistant application142 on a remote data processing system 102 via the network 146. Forexample, the client device 132 can include the functionalities of theNLP component 106 and access the remainder of the components of thedigital assistant application 142 via the network 146 to the dataprocessing system 102. The vehicle 144 can include and execute aseparate instance of the components of the data processing system 102,such as the digital assistant application 142, the configuration trackercomponent 116, the application selector component 118, and theperformance evaluator component 120. For example, the digital assistantapplication 142 may be pre-installed on a head unit of a vehicle 144.The digital assistant applications 142 accessible or executing on theclient devices 132 and the vehicles 144 may each have differentfunctionalities.

The client devices 132 and the vehicles 144 can each include at leastone logic device such as a computing device having a processor tocommunicate with each other with the data processing system 102 via thenetwork 146. The client devices 132 and the vehicles 144 can include aninstance of any of the components described in relation to the dataprocessing system 102. The client devices 132 and the vehicles 144 caninclude an instance of the digital assistant application 142. The clientdevices 132 can include a desktop computer, laptop, tablet computer,personal digital assistant, smartphone, mobile device, portablecomputer, thin client computer, virtual server, speaker-based digitalassistant, or other computing device. The vehicle 144 can be a car,truck, motorcycle, airplane, blimp, boat, submarine, or othertransportation device. The vehicle 144 can include one or moreprocessors that can execute an instance of the digital assistantapplication 142 or any component thereof. The processors can be acomponent of the head unit disposed in the vehicle 144, such as a headunit in an automobile.

The components of the system 100 can communicate over a network 146. Insome implementations, one or more of the client devices 132 can belocated within an instance of the vehicle 144. For example, the clientdevices 132 can be the mobile phone of a driver driving the vehicle 144.In some implementations, one or more of the client devices 132 can beremote to the vehicles 144. For example, after the driver parks andexits the vehicle 144 for work, the driver's mobile phone may be remoteto the vehicle 144.

The network 146 can include, for example, a point-to-point network, abroadcast network, a wide area network, a local area network, atelecommunications network, a data communication network, a computernetwork, an ATM (Asynchronous Transfer Mode) network, a SONET(Synchronous Optical Network) network, a SDH (Synchronous DigitalHierarchy) network, an NFC (Near-Field Communication) network, a localarea network (LAN), a wireless network or a wireline network, andcombinations thereof. The network 146 can include a wireless link, suchas an infrared channel or satellite band. The topology of the network146 may include a bus, star, or ring network topology. The network 146can include mobile telephone networks using any protocol or protocolsused to communicate among mobile devices, including advanced mobilephone protocol (AMPS), time division multiple access (TDMA),code-division multiple access (CDMA), global system for mobilecommunication (GSM), general packet radio services (GPRS), or universalmobile telecommunications system (UMTS). Different types of data may betransmitted via different protocols, or the same types of data may betransmitted via different protocols.

The network 146 can include a short-range communication link (e.g.,ranging up to 30 meters) established between the client devices 132 andthe vehicles 144, using Bluetooth, Bluetooth Low Energy, DedicatedShort-Range Communications (DSRC), or Near-Field Communications (NFC)protocols. Using such protocols, the data processing system 102 canestablish a communication link with one or more client devices 132 viathe interface 104. The data processing system 102 can establish acommunication link with one or more vehicles 144 via the interface 104.The short-range communication link may be established between the clientdevices 132 and the vehicles 144 via pairing protocol.

The client devices 132 can include sensors 134, speakers 136, peripheraldevices 138, interfaces 104, and transducers 140. The client devices 132can execute an instance of the digital assistant application 142. Thevehicles 144 can include sensors 134, speakers 136, peripheral devices138, interfaces 104, and transducers 140. The vehicles 144 can executean instance of the NLP component 106. The peripheral devices 138 for theclient device 132 can include user input/output devices, such as akeyboard, a monitor, and headphones, among others. The peripheraldevices 138 for the vehicle 144 can include any other devices, such as agarage door opener, a window opener, a passenger door opener, a trunkopener, a multimedia device player, and a temperature control, amongothers.

The client devices 132 and the vehicle 144 can include at least onesensor 134, at least one transducer 140, at least one audio driver, andat least one speaker 136. The sensor 134 can include a microphone oraudio input sensor. The sensor 134 can also include at least one of aGPS sensor, proximity sensor, ambient light sensor, temperature sensor,motion sensor, accelerometer, or gyroscope. The sensor can include anoccupancy or weight sensor. The transducer 140 can convert the audioinput into an electronic signal. The audio driver can include a scriptor program executed by one or more processors of the client devices 132or the vehicle 144 to control the speaker 136. The speaker 136 canrender audio signals by converting electrical signals into audiblewaves.

The client devices 132 and the vehicle 144 can be associated with an enduser that enters voice queries as input audio signals into the clientdevices 132 or the vehicle 144 (via the sensor 134) and receives audiooutput in the form of a computer generated voice that can be providedfrom the data processing system 102. In response to the input audiosignals, the client devices 132 and the vehicle 144 can also receiveaction data structures to perform predetermined functions or actions.The interface 104 can receive or provide data messages to the directaction API 112 of the data processing system 102 and enablecommunication between the components of the system 100. The clientdevices 132 and the vehicle 144 can also include a user interface thatenables a user to interact with the components of the system 100.

The data processing system 102 can include at least one server having atleast one processor. For example, the data processing system 102 caninclude a plurality of servers located in at least one data center orserver farm. The data processing system 102 can determine from an audioinput signal a request and a trigger keyword associated with therequest. Based on the request and trigger keyword, the data processingsystem 102 can generate or select response data. The response data canbe audio-based or text-based. For example, the response data can includeone or more audio files that, when rendered, provide an audio output oracoustic wave. The data within the response data can also be referred toas content items. The response data can include other content (e.g.,text, video, or image content) in addition to audio content.

The data processing system 102 can include multiple, logically groupedservers and facilitate distributed computing techniques. The logicalgroup of servers may be referred to as a data center, server farm, or amachine farm. The servers can be geographically dispersed. A data centeror machine farm may be administered as a single entity, or the machinefarm can include a plurality of machine farms. The servers within eachmachine farm can be heterogeneous—one or more of the servers or machinescan operate according to one or more type of operating system platform.The data processing system 102 can include servers in a data center thatare stored in one or more high-density rack systems, along withassociated storage systems, located for example in an enterprise datacenter. In this way, the data processing system 102 with consolidatedservers can improve system manageability, data security, the physicalsecurity of the system, and system performance by locating servers andhigh performance storage systems on localized high performance networks.Centralization of all or some of the data processing system 102components, including servers and storage systems, and coupling themwith advanced system management tools allows more efficient use ofserver resources, which saves power and processing requirements andreduces bandwidth usage. Each of the components of the data processingsystem 102 can include at least one processing unit, server, virtualserver, circuit, engine, agent, appliance, or other logic device such asprogrammable logic arrays configured to communicate with the datarepository 122 and with other computing devices.

The data processing system 102 can include the data repository 122. Thedata repository 122 can include one or more local or distributeddatabases and can include a database management system. The datarepository 122 can include computer data storage or memory and can storeone or more parameters 124, one or more policies 126, response data 128,and templates 130, among other data. The parameters 124, policies 126,and templates 130 can include information such as rules about a voicebased session between the client devices 132, the data processing system102, and the vehicle 144. The parameters 124, policies 126, andtemplates 130 can also include information for another digital assistantapplication 142 received via the interface 104 from another source(e.g., the data processing system 102, the client devices 132, and thevehicles 144). For example, the parameters 124, policies 126, andtemplates 130 stored in the data repository 122 of a digital assistantapplication 142 hosted on a vehicle 144 can include the parameters 124,policies 126, and templates 130 from the data repository 122 of adigital assistant application 142 accessible via the client device 132,and vice-versa. In this manner, the parameters 124, policies 126, andtemplates 130 of different digital assistant applications 142 may beshared and used by one another. The response data 128 can includecontent items for audio output or associated metadata, as well as inputaudio messages that can be part of one or more communication sessionswith the client devices 132.

An application, script, program, or other components that are associatedwith the data processing system 102 can be installed at the clientdevices 132 or the vehicle 144. The application can enable the clientdevices 132 or vehicle 144 to communicate input audio signals (and otherdata) to the interface 104 of the data processing system 102. Theapplication can enable the client devices 132 and the vehicle 144 todrive components of the client devices 132 and the vehicle 144 to renderthe output audio signals.

The NLP component 106 can receive input audio signals. The dataprocessing system 102 can receive the input audio signal from the clientdevices 132 or the vehicle 144 (e.g., via the transducers 140) includedin a data packet. A first device can execute the NLP component 106, andthe NLP component 106 can receive the input audio signal from a seconddevice. For example, a client device 132 can receive the input audiosignal from and the data processing system 102 can execute the NLPcomponent 106.

The NLP component 106 can convert input audio signals into recognizedtext by comparing the input audio signal against a stored,representative set of audio waveforms and choosing the closest matches.The representative waveforms can be generated across a large set ofinput audio signals. Once the input audio signal is converted intorecognized text, the NLP component 106 can match the text to words thatare associated, for example, via a learning phase, with actions oroutput audio signals.

From the input audio signal, the NLP component 106 can identify at leastone request or at least one command corresponding to the request. Therequest can indicate intent or subject matter of the input audio signal.The command can indicate a type of action likely to be taken. Forexample, the NLP component 106 can parse the input audio signal toidentify at least one request to open the vehicle's windows or skip to anext audio file in a music playlist. The command can include at leastone word, phrase, root or partial word, or derivative indicating anaction to be taken. The request can also include a trigger keyword. Thetrigger keyword may be associated with a particular digital assistantapplication accessible via the client device 132 or running on thevehicle 144. For example, the trigger keywords “go” may be associatedwith the digital assistant application accessible via the client device132, and “ok” may be associated with the digital assistant applicationrunning on the vehicle 144. Which digital assistant application isselected to execute the indicated command is detailed herein.

The NLP component 106 can also determine whether the at least onecommand corresponding to the request of the input audio signal relatesto a vehicle functionality of the vehicle 144 or to a non-vehiclefunctionality of the client device 132. Upon extraction of the intent,subject matter, and the command from the input audio signal, the NLPcomponent 106 can compare the word, phrases, root or partial words, orderivatives to a semantic knowledge graph. The semantic knowledge graphmay specify a set of words, phrases, root or partial words, orderivatives relating to a vehicle functionality and non-vehiclefunctionality. For example, “windshield” may be marked as related to avehicle functionality and “phonebook” may be marked as related to anon-vehicle functionality. The semantic knowledge graph may also specifya relationship between each word, phrase, root or partial word, orderivative. The semantic knowledge graph may be maintained at the datarepository 122. Based on the comparison with the semantic knowledgegraph, the NLP component 106 can determine whether the word, phrase,root or partial word, or derivative of the command relates to a vehiclefunctionality or a non-vehicle functionality.

The response selector component 114 can obtain information from the datarepository 122 where it can be stored as part of the response data 128.The response selector component 114 can query the data repository 122 toselect or otherwise identify response phrases or content items, e.g.,from the response data 128.

The audio signal generator component 110 can generate or otherwiseobtain an output signal that includes the content item. The dataprocessing system 102 can execute the audio signal generator component110 to generate or create an output signal corresponding to the contentitem or request. For example, once a request is fulfilled, the audiosignal generator component 110 can generate an audio output signal thatincludes the phrase “The action was completed.”

The interface 104 can be a data interface or a network interface thatenables the components of the system 100 to communicate with oneanother. The interface 104 of the data processing system 102 can provideor transmit one or more data packets that include the action datastructure, audio signals, or other data via the network 146 to theclient devices 132 or vehicle 144. For example, the data processingsystem 102 can provide the output signal from the data repository 122 orfrom the audio signal generator component 110 to the client devices 132.The data processing system 102 can also instruct, via data packettransmissions, the client devices 132 or the vehicle 144 to perform thefunctions indicated in the action data structure. The output signal canbe obtained, generated, transformed to, or transmitted as one or moredata packets (or other communications protocol) from the data processingsystem 102 (or other computing device) to the client devices 132 or thevehicle 144.

The direct action API 112 of the data processing system 102 cangenerate, based on, for example, the request, action data structures.The action data structure can include data or instructions for theexecution of a specified action to satisfy the request. In someimplementations, the action data structure can be a JSON formatted datastructure or an XML formatted data structure. The direct action API 112may be prevented from carrying out the action specified in the requestuntil invocation by the application selector component 118. The detailsof the functionalities of the application selector component 118 inrelation to the direct action API 112 are explained below.

Depending on the action specified in the request, the direct action API112 can execute code or a dialog script that identifies the parametersrequired to fulfill the request. The action data structures can begenerated responsive to the request. The action data structure can beincluded in messages that are transmitted to or received by the clientdevices 132 or the vehicle 144. Based on the request parsed by the NLPcomponent 106, the direct action API 112 can determine to which of theclient devices 132 or the vehicles 144 the message should be sent forprocessing. Whether the client device 132 or the vehicle 144 is selectedto process the message is detailed below with respect to the applicationselector component 118. The direct action API 112 can package therequest into an action data structure for transmission to the vehicle144. The direct action API 112 can access a vehicle ID from the responsedata 128 to determine which vehicle is associated with the user thatgenerated the request. Once received, the vehicle 144 can process theaction data structure and can perform the indicated action. The directaction API 112 can also package the request into an action datastructure for execution by the client device 132. Once received, theclient device 132 can process the action data structure using thedigital assistant application 142 or one or more applications running onthe client device 132.

The action data structure can include information for completing therequest. For example, the action data structure can be an XML or JSONformatted data structure that includes attributes used in completing orotherwise fulfilling the request. The attributes can include a locationof the vehicle 144, a location of the client devices 132, anauthorization level of a user associated with a client devices 132, avehicle identifier, an interface identifier, a vehicular state, or arequest state. In some implementations, the request state includes oneor more attributes that should be satisfied before the action isfulfilled. For example, with the request “Ok, change the song,” therequest state may have the attribute {requestor[authorized, passenger]},indicating that the request should be an explicitly authorized user or apassenger in the vehicle.

The direct action API 112 can retrieve a template 126 from therepository 122 to determine which fields or attributes to include in theaction data structure. The direct action API 112 can determine necessaryparameters and can package the information into an action datastructure. The direct action API 112 can retrieve content from therepository 122 to obtain information for the attributes of the datastructure.

The direct action API 112 can populate the fields with data from theinput audio signal. The direct action API 112 can also populate thefields with data from the client devices 132 or the vehicle 144, or fromanother source. The direct action API 112 can prompt a user foradditional information when populating the fields. The templates 130 canbe standardized for different types of actions, such as playing mediafiles through the vehicle's head unit, responding to messages, andperforming functions within the vehicle 144. The action data structurecan initially be generated by a direct action API 112 executed by aremote data processing system 102. The remote data processing system 102can transmit the action data structure to the data processing system 102of the vehicle 144, which can add fields and attributes to the actiondata structure.

The direct action API 112 can obtain response data 128 (or parameters124 or policies 126) from the data repository 122, as well as datareceived with end user consent from the client devices 132 to determinelocation, time, user accounts, and logistical or other information inorder to reserve a car from the car share service. The response data 128(or parameters 124 or policies 126) can be included in the action datastructure. When the content included in the action data structureincludes end user data that is used for authentication, the data can bepassed through a hashing function before being stored in the datarepository 122.

The data processing system 102 can include, interface, or otherwisecommunicate with the configuration tracker component 116. Theconfiguration tracker component 116 can be hosted on and executed fromthe vehicle 144. The configuration tracker component 116 can identify afunctionality of each digital assistant application. The configurationtracker component 116 can identify each digital assistant application142 running on or otherwise accessible via a client device 132. Theconfiguration tracker component 116 can identify each digital assistantapplication 142 running on the vehicle 144. The client device 132 may becommunicatively coupled (e.g., via network 146 or via a short-rangecommunication link) to the data processing system 102 or the vehicle144. With the communication link established, each digital assistantapplication 142 accessible from the vehicle 144, such as the digitalassistant application 142 running on the vehicle 144 and the digitalassistant application 142 accessible via the client device 132, may beprevented from executing the action specified by the command untilinvoked by the application selector component 118. In addition, thetransducer 140 at the client device 132 may also be disabled until thedigital assistant application 142 is invoked by the application selectorcomponent 118. Details of the functionality of the application selectorcomponent 118 are explained below.

The functionalities of the digital assistant application 142 hosted onthe vehicle 144 (sometimes referred to as a “primary digital assistantapplication”) may differ from the functionalities of the digitalassistant application 142 accessible via the client device 132(sometimes referred to as a “secondary digital assistant application”).For example, the digital assistant application 142 installed on thevehicle 144 may activate various function on the vehicle 144, such asopening a door, turning on a windshield, or changing temperaturecontrols. On the other hand, the digital assistant application 142accessible through the client device 132 may carry out other functions,such as making a telephone call or accessing a calendar. Forfunctionalities of the digital assistant application 142 accessible viathe client device 132 that do not differ from the functionalities of thedigital assistant application 142 hosted on the vehicle 144, one of thedigital assistant applications 142 may be able to execute thefunctionality faster or more efficiently.

The configuration tracker component 116 can identify the functionalitiesof each digital assistant application 142 accessible via the clientdevice 132. There may be multiple digital assistant applications 142accessible via multiple client devices 132. The configuration trackercomponent 116 can identify an application profile of the digitalassistant application 142. The application profile may include metadatafor the digital assistant application 142 and may indicate a name and atype for the application. The configuration tracker component 116 canidentify a version number of the digital assistant application 142. Theconfiguration tracker component 116 can identify a connectivity of theclient device 132 to the vehicle 144 or to an external network (e.g.,the network 146) for accessing the remote data processing system 102.The configuration tracker component 116 can identify one or moreroutines or processes (e.g., application programming interface (API)calls) supported or used by the digital assistant application 142. Theconfiguration tracker component 116 can identify one or more peripheraldevices 138 interfacing with the client device 132. The one or moreperipheral devices 138 may include input/output devices, such as amonitor, keyboard, and card-reader, among others, connected to theclient device 132. The configuration tracker component 116 can identifyconsumption of computing resources at the client device 132 (e.g., CPUtime, remaining battery life, or memory use, for example) through whichthe digital assistant application 142 is accessed.

The configuration tracker component 116 can identify the functionalitiesof the digital assistant application 142 hosted on the vehicle 144. Theconfiguration tracker component 116 can identify an application profileof the digital assistant application 142. The application profile mayinclude metadata for the digital assistant application 142 and mayindicate a name and a type for the application. The configurationtracker component 116 can identify a version number of the digitalassistant application 142. The version number of the digital assistantapplication 142 hosted on the vehicle 144 may differ from the versionnumber of the digital assistant application 142 accessible via theclient device 132. The configuration tracker component 116 can identifya connectivity of the vehicle 144 to the client device 132 or to anexternal network (e.g., the network 146) for accessing the remote dataprocessing system 102. The configuration tracker component 116 canidentify one or more routines or processes (e.g., applicationprogramming interface (API) calls) supported or used by the digitalassistant application 142. The configuration tracker component 116 canidentify one or more peripheral devices 138 interfacing with the vehicle144. The one or more peripheral devices 138 may include various devicesin the vehicle 144, such as a temperature control, passenger dooropener, trunk opener, garage door opener, a steering wheel, andmultimedia device, among others accessible by the digital assistantapplication 142 running from the vehicle 144. The configuration trackercomponent 116 can identify consumption of computing resources (e.g., CPUtime, remaining battery life, memory use, etc.) at the vehicle 144hosting the digital assistant application 142.

The data processing system 102 can include, interface, or otherwisecommunicate with the application selector component 118. The applicationselector component 118 can be hosted on and executed from the vehicle144. The application selector component 118 can determine whether thefunctionality of the digital assistant application 142 accessible viathe client device 132 or the functionality of the digital assistantapplication 142 hosted on the vehicle 144 can support the commandcorresponding to the request. The application selector component 118 canidentify an action specified by the command as processed by the NLPcomponent 106. The application selector component 118 can parse theaction data structure generated by the direct action API 112 to identifythe specified action.

To determine that the specified action is (or is not) supported by thedigital assistant application 142, the application selector component118 can compare the operational capabilities or functionality of thedigital assistant application 142 accessible via the client device 132to the action specified by the command. The application selectorcomponent 118 can also compare the operational capabilities orfunctionality of the digital assistant application 142 hosted on thevehicle 144. The application selector component 118 can determine thatthe functionality of the digital assistant application 142 can supportthe action specified by the command based on combination of factors. Theapplication selector component 118 can determine that the actioncorresponding to one or more routines or processes of the digitalassistant application 142 corresponding to the action specified by thecommand. Responsive to the determination, the application selectorcomponent 118 can determine that the functionality of the digitalassistant application 142 supports the action. The application selectorcomponent 118 can determine that the action specified by the command isperformed via an external network. Responsive to the determination, theapplication selector component 118 can determine that the functionalityof the digital assistant application 142 supports the action when thereis access to the external network (e.g., network 146). The applicationselector component 118 can determine the action specified by the commandis performed via a particular peripheral device 138. Responsive to thedetermination, the application selector component 118 can identifywhether the particular peripheral device 138 is interfacing with theclient device 132, and the particular peripheral devices 138 isinterfacing with the vehicle 144. The application selector component 118can determine the action specified by the command is performed by aparticular type of application (e.g., the digital assistant application142 at the vehicle 144 or the digital assistant application 142 on theclient device 132). Responsive to the determination, the applicationselector component 118 can determine whether the application profile ofthe digital assistant application 142 is a type of application that cancarry out the specification action.

Based on the determination of the functionality of the respectivedigital applications 142, the application selector component 118 canselect one of the digital assistant application 142 accessible via theclient device 132 or the functionality of the digital assistantapplication 142 hosted on the vehicle 144. For example, the applicationselector component 118 can set the digital assistant application 142hosted together on the vehicle 144 as the default digital assistantapplication and set the digital assistant application 142 accessible viathe client device 132 as the secondary digital assistant application. Inthe absence of the secondary digital assistant application, theapplication selector component 118 can select the primary digitalassistant application to carry out the command. The application selectorcomponent 118 can determine the functionality of the digital assistantapplication 142 accessible via the client device 132 can support theaction specified by the command. Responsive to the determination, theapplication selector component 118 can select the digital assistantapplication 142 accessible via the client device 132. The applicationselector component 118 can determine that there are multiple digitalassistant applications 142 accessible via the one or more client devices132. Responsive to the determination that there are multiple digitalassistant applications 14, the application selector component 118 canselect the one digital assistant application 142 accessible via theclient device 132 whose functionality can support the action specifiedby the command. Conversely, the application selector component 118 candetermine the functionality of the digital assistant application 142hosted on the vehicle 144 but not that of the digital assistantapplication 142 accessible via the client device 132 can support theaction specified by the command. Responsive to the determination, theapplication selector component 118 can select the digital assistantapplication 142 hosted on the vehicle 144.

When both functionalities of the digital assistant applications 142accessible via the client device 132 and hosted on the vehicle 144 aredetermined support the action specified by the command, the digitalassistant application 142 can select based on which digital assistantapplication 142 can more quickly carry out the action. The applicationselector component 118 can compare the version number of the digitalassistant application 142 hosted on the vehicle 144 with the versionnumber of the digital assistant application 142 accessible via theclient device 132. The application selector component 118 can select thedigital assistant application 142 with the higher version number, as thecorresponding the digital assistant application 142 may be moreup-to-date. The application selector component 118 can compare thecomputing resources consumption at the vehicle 144 with the computingresources consumption at the client device 132. The application selectorcomponent 118 can select the client device 132 or the vehicle 144 basedon which has the lower computing resource consumption, as the actionspecified by the command is likely to be executed faster when computingresource consumption is lower.

The application selector component 118 can also select the digitalassistant application 142 accessible via the client device 132 or thefunctionality of the digital assistant application 142 hosted on thevehicle 144 based on other factors. These other factors can override theselection based on the determination on which digital assistantapplication 142 can support the action specified by the command. Theapplication selector component 118 can also select the digital assistantapplication 142 accessible via the client device 132 or the digitalassistant application 142 hosted on the vehicle 144 based on an inputvia a vehicle input interface. The vehicle input interface may be one ofthe peripheral devices 138 connected to the vehicle 144, such as abutton on a steering wheel or a touchscreen on a center stack in thevehicle 144. For example, the selection may be based on a length of abutton press on the button on the steering wheel of the vehicle 144. Theapplication selector component 118 can determine that the length of thebutton press is greater than or equal to a predetermined threshold(e.g., 5 seconds). In response to the determination, the applicationselector component 118 can select the digital assistant application 142hosted on the vehicle 144. The application selector component 118 candetermine that the length of the button press is shorter than thepredetermined threshold. In response to this determination, theapplication selector component 118 can select the digital assistantapplication 142 accessible via the client device 132. A position of theinteraction on the touchscreen may also be used to select the digitalassistant application 142 accessible via the client device 132 or thedigital assistant application 142 hosted on the vehicle 144.

The application selector component 118 can also select the digitalassistant application 142 accessible via the client device 132 or thefunctionality of the digital assistant application 142 hosted on thevehicle 144 based on the trigger keyword recognized in the audio inputsignal. The application selector component 118 can compare the triggerkeyword to a set of keywords associated with corresponding digitalassistant applications 148. Based on the comparison of the trigger worddetected from the audio input signal with the set of associatedkeywords, the application selector component 118 can select thecorresponding digital assistant application 148. For example, there maybe two client devices 132 in the vehicle 144 with respective digitalassistant applications 148 associated with different trigger keywords.The trigger word “okay” may be associated with the digital assistantapplication 148 on a first client device 132 and the trigger word “hey”may be associated with the digital assistant application 148 on a secondclient device 132. Using the set of keywords, when the audio inputsignal including “hey” is received, the application selector component118 can select the digital assistant application 148 on the secondclient device 132.

The application selector component 118 can also select the digitalassistant application 142 accessible via the client device 132 or thefunctionality of the digital assistant application 142 hosted on thevehicle 144 based on whether the command is related to a vehicularfunction. As explained above, the NLP component 106 can determinewhether the command is related to a vehicular or non-vehicular functionusing a semantic knowledge graph. Using the determination, theapplication selector component 118 can select the corresponding digitalassistant application 142. The application selector component 118 candetermine that the command is related to a vehicular function. Inresponse to the determination, the application selector component 118can select the digital assistant application 142 hosted on the vehicle144. The application selector component 118 can determine that thecommand is determined to relate to a non-vehicular function. In responseto the determination, the application selector component 118 can selectthe digital assistant application 142 accessible via the client device132. The application selector component 118 can determine that there aremultiple digital assistant applications 142 accessible via the one ormore client devices 132. In response to the determination, theapplication selector component 118 can select the one digital assistantapplication 142 accessible via the client device 132 whose functionalitycan support the action specified by the command identified to be relatedto a non-vehicular function.

The application selector component 118 can determine that there aremultiple digital assistant applications 142 accessible whosefunctionalities can support the action specified by the command. Inresponse to the determination, the application selector component 118can select one of the digital assistant applications 142 based on aquality score. These multiple digital assistant applications 142 caninclude the digital assistant applications 142 accessible via the one ormore client devices 132 and the digital assistant application 142 hostedon the vehicle 144. The data processing system 102 can include,interface, or otherwise communicate with the performance evaluatorcomponent 120. The performance evaluator component 120 can determine thequality score for each digital assistant application 142 accessible viaone or more client devices 132 based on any number of factors. Theperformance evaluator component 120 can determine the quality scorebased on a feedback indicator. The feedback indicator may be a numericalvalue representing a user's response to a prompt inquiring about qualityof interaction with the digital assistant application 142. To obtain thefeedback indicator, the digital assistant application 142 accessible viathe client device 132 can display the prompt inquiring about quality ofinteraction on the client device 132 displaying the digital assistantapplication 142. The digital assistant application 142 hosted on thevehicle 144 or the performance evaluator component 120 can also displaythe prompt inquiring about quality of interaction on the vehicle 144displaying the digital assistant application 142. The prompt may includeone or more user interface elements (e.g., buttons, radio buttons,checkboxes, etc.), each indicating the quality of interaction. Once aninteraction event is detected on one of the user interface elements onthe prompt, the performance evaluator component 120 can identify thefeedback indicator corresponding to the quality of interaction. Theperformance evaluator component 120 can determine the quality scoreusing the feedback indicator.

The performance evaluator component 120 can also determine the qualityscore for each digital assistant application 142 based on a rating scorefrom an application distribution platform. The application distributionplatform can run on an external server accessible via the network 146.The external server can host applications, such as the digital assistantapplications 142, for download by the client device 132 or the vehicle144. The external server for the application distribution platform maystore the rating scores for all the digital assistant applications 142.From the application distribution platform running on the externalserver, the performance evaluator component 120 can identify the ratingscore for each digital assistant application 142 identified asaccessible via the client device 132 or hosted on the vehicle 144. Theperformance evaluator component 120 can send a request for the ratingscore to the external server, when there is connectivity with thenetwork 146. Using the rating score for each digital assistantapplication 142, the performance evaluator component 120 can determinethe quality score for the corresponding digital assistant application142.

The performance evaluator component 120 can also run an automated testevaluation to determine the quality score for the corresponding digitalassistant application 142. The automated test evaluation may be executedusing an application or the performance evaluator component 120interfacing with the digital assistant application 142. The automatedtest evaluation may include a set of audio input signal stimuli and acorresponding expected action data structure for the audio input signalstimuli. The automated test evaluation may be run simultaneous to theexecution of the digital assistant application 142. While executing, theautomated test evaluation can feed each audio input signal stimulus intothe digital assistant application 142. The automated test evaluation canidentify an action data structure generated by the digital assistantapplication 142. The automated test evaluation can compare the actiondata structure generated by the digital assistant application 142 withthe expected action data structure. From the comparison, the automatedtest evaluation can identify a number of matches and a number ofmismatches between the two action data structures. Based on the numberof matches and the number of mismatches, the performance evaluatorcomponent 120 can determine the quality score for the digital assistantapplication 142.

Once the quality score for each digital assistant application 142 isdetermined, the application selector component 118 can select thedigital assistant application 142 with the highest quality score havingthe functionality to support the action specified by the command. Thequality score may be based on any combination of the feedback indicator,the rating score, and the automated test evaluation. The applicationselector component 118 can identify a subset of digital assistantapplications 142 whose functionality can support the action specified bythe command. From the subset of digital assistant applications 142, theapplication selector component 118 can identify the digital assistantapplication 142 with the highest quality score as determined by theperformance evaluator component 120.

The application selector component 118 can invoke the digital assistantapplication 142 selected based on the determination that thefunctionality of the digital assistant application 142 can support theaction specified by the command. To invoke the selected digitalassistant application 142, the application selector component 118 canidentify the action data structure corresponding to the command. Theapplication selector component 118 can call the direct action API 112 ofthe selected digital assistant application 142 to execute and processthe action data structure corresponding to the command in the audioinput signal. The application selector component 118 can determine thatthe digital assistant application 142 selected is accessible via theclient device 132. Responsive to the determination, the applicationselector component 118 can assign control of the input interfaces (e.g.,the sensor 134, the interface 104, the transducer 140, and otherperipheral devices 138) of the vehicle 144 to the client device 132. Forexample, the command may be to play an audio file on the client device132, and the digital assistant application 142 selected is accessiblevia the client device 132. To carry out the command, the applicationselector component 118 can assign control of the speakers 136 from thevehicle 144 to the digital assistant application 142 accessible via theclient device 132 (sometimes referred to in such instances as a“brought-in digital assistant application”), such that the audio isplayed from the speakers 136 of the vehicle 144.

In invoking the selected digital assistant application 142, theapplication selector component 118 can use application data from anon-selected digital assistant application 142. The application selectorcomponent 118 can access the data repository 122 of the non-selecteddigital assistant application 142 to retrieve the parameters 124, thepolicies 126, the response data 128, the templates 130, and otherinformation (e.g., phone numbers, calendar events, etc.). For example,when the action specified by the command is to call a particular phonenumber, the application selector component 118 can retrieve the phonenumber from the non-selected digital assistant application 142 (oranother application) running on a first client device 132. Theapplication selector component 118 can store the retrieved applicationdata in the local data repository 122. Prior to invocation, theapplication selector component 118 can modify the action data structureto include the application data retrieved from the non-selected digitalassistant application 142. The application selector component 118 canthen call the direct action API 112 of the selected digital assistantapplication 142 using the direct action structure modified by theapplication data retrieved from the non-selected digital assistantapplication 142. Continuing with the previous example, the applicationselector component 118 can call the selected digital assistantapplication 142 accessible via a second client device 132 to make thephone call using the phone number retrieved from the first client device132.

FIG. 2 illustrates a top view of the vehicle 144 and illustrates theinterior cabin of the vehicle 144. In the vehicular environment 200, theinterior cabin of the vehicle 144 can include four seats, a plurality ofspeakers 136, and two client devices 132(1) and 132(2). The two clientdevices 132(1) and 132(2) may have been carried in by one or morepassengers of the vehicle 144. The vehicle 144 can also include a headunit 202. The head unit 202 can execute one or more of the componentsdescribed in relation to the data processing system 102, such as adigital assistant application 142(1), the configuration trackercomponent 116, the application selector component 118, and theperformance valuator component 120. The client device 132(1) can executeor make accessible one or more of the components described in relationto the data processing system 102, such a digital assistant application142(2). The client device 132(2) also can execute or make accessible oneor more of the components described in relation to the data processingsystem 102, such a digital assistant application 142(3). The clientdevices 132(1) and 132(2) may have established communication links(e.g., via pairing) with the head unit 202.

Referring to FIG. 1 among others, a transducer 140 within the vehicle144 may detect an audio input signal uttered by a passenger within thevehicle 144. Upon detection of the audio input signal, the NLP component106 operating from the head unit 202 of the vehicle 144 can parse theaudio input signal to identify a command. Using the command parsed fromthe audio input signal, the direct action API 112 running in the headunit 202 of the vehicle 144 can generate an action data structure. Theaction data structure can include data or instructions for the executionof a specified action to satisfy the command.

The configuration tracker component 116 operating from the head unit 202of the vehicle 144 can identify each digital assistant application141(1)-(3) running on or in communication with the head unit 202. Thedigital assistant applications 141(1)-(3) may have differingfunctionalities. For example, the digital assistant application 141(1)running in the head unit 202 may support vehicle-related commands (e.g.,“pop the trunk”) while the digital assistant application 141(2) of theclient device 132(1) and the digital assistant application 141(2) of theclient device 132(2) may support non-vehicle related commands (e.g.,“play a song”). The configuration tracker component 116 can identify oneor more functionalities of each of the identified digital assistantapplications 141(1)-(3), such as the application profile, versionnumber, connectivity to the network 146 or the vehicle 144, peripheraldevices 138, consumption of computing resources, and supported routines,among others.

Having identified the functionalities of the digital assistantapplications 141(1)-(3), the application selector component 118 candetermine which digital assistant applications 141(1)-(3) can supportthe action specified by the command. The application selector component118 can select the digital assistant application 141(1)-(3) based onwhich can support the action specified by the command. When two or moreof the digital assistant application 141(1)-(3) can support the actionspecified by the command, the application selector component 118 canselect one of the digital assistant applications 141(1)-(3) using otherfactors, such as the version number and consumption of computingresources, among others. Once determined, the application selectorcomponent 118 can invoke the selected digital assistant application141(1)-(3) by calling the respective direct action API 112 to execute inaccordance with the generated action data structure. When applicationdata from one digital assistant application 141(2) is to be used by theselected digital assistant application 141(3), the application data maybe transferred from the selected digital assistant application 141(2) tothe non-selected digital assistant application 141(3). The applicationselector component 118 can modify the data structure using thetransferred data structure. The direct action API 112 may be called tocarry out the action specified by the command.

FIG. 3 illustrates an example method 300 to selectively invokeapplications for execution. The method 300 may be implemented orexecuted by the system 100 described above in conjunction with FIGS. 1and 2 or system 400 detailed below in conjunction with FIG. 4 . Themethod 300 can include receiving an audio signal (BLOCK 305). The method300 can include parsing the audio signal to identify a command (BLOCK310). The method 300 can include generating an action data structure(BLOCK 315). The method 300 can include identifying a functionality of adigital assistant application (BLOCK 320). The method 300 can includedetermining that the functionality supports the command (BLOCK 325). Themethod 300 can include selecting the digital assistant application(BLOCK 330). The method 300 can include invoking the selected digitalassistant application (BLOCK 335).

The method 300 can include receiving an audio signal (BLOCK 305). A dataprocessing system can receive the input audio signal. For example, anNLP component, executed by the data processing system, can receive theinput audio signal. The data processing system (and the NLP component)can be a component of or otherwise executed by a client device, avehicle, or be a standalone device. The sensor, such as a microphone, atthe client device or the vehicle can detect the input audio signal andthen the respective client device or vehicle can transmit the inputaudio signal to the data processing system. For example, an applicationexecuted on the client device can detect a user speaking “Ok, open thesunroof.” The detected utterance can be encoded into an input audiosignal and transmitted to the NLP component of the data processingsystem or vehicle.

The method 300 can include parsing the audio signal to identify acommand (BLOCK 310). The NLP component can parse the input audio signalto identify a request in the input audio signal. The NLP component canidentify a vehicle associated with the request. The NLP component canidentify a fulfillment interface associated with the request and thevehicle. The fulfillment interface can be the interface of one of theclient device or the vehicle that will execute the action data structureto fulfill the request of the input audio signal.

In the above example, the request can be to open the sunroof. In thisexample, the fulfillment interface can be the interface of the vehiclethat includes the sunroof. In some implementations, the vehicle can beexplicitly stated in the input audio signal. For example, a user mayassign nicknames to his vehicle (e.g., the user may name his red ToyotaCamry the “red car”). When explicitly stated, the input audio signalcould be “OK, open the sunroof of my red car.” In some implementations,the data processing system can determine which vehicles associated withthe user's account can fulfill the action. For example, the user's redcar may include a sun roof and the user's blue car may not. Afterreceiving the input audio signal “Ok, open the sunroof.” The dataprocessing system may automatically select the user's red car. In someimplementations, the data processing system may ask the user forconfirmation of the vehicle.

The method 300 can include generating an action data structure (BLOCK315). The direct action API can generate the data structure that can betransmitted and processed by the client device or the vehicle to fulfilthe request of the input audio signal. For example, continuing the aboveexample, the direct action API can generate a first action datastructure for opening the sunroof of the user's car. The directionaction API can generate the action data structure using a templateretrieved from the data processing system's data repository. The actiondata structure can include fields used to fulfill the request. Forexample, for the request to open the sunroof, the action data structurecan include a field (or attribute) for the vehicle ID to which theaction data structure should be transmitted when the action datastructure is approved.

The method 300 can include identifying a functionality of a digitalassistant application (BLOCK 320). For example, a configuration trackercomponent can identify each digital assistant application in thevehicle. The digital assistant applications can be hosted on the vehicleitself or may be accessible via the client device. All of the digitalassistant applications in the vehicle may be communicatively linked viashort-range communications. For each digital assistant applicationidentified, the configuration tracker component can identify thefunctionality of the digital assistant application, such as theapplication profile, version number, connectivity to the network 146 orthe vehicle 144, peripheral devices 138, consumption of computingresources, and supported routines, among others.

The method 300 can include determining that the functionality supportsthe command (BLOCK 325). For instance, an application selector componentcan determine whether the functionality identified for the correspondingdigital assistant application supports the action specified by thecommand. Some commands may be related to a vehicle function (e.g., “openthe sunroof”), while other commands may be related to a non-vehiclefunction (e.g., “call John”). The determination whether the commandrelates to a vehicle function or a non-vehicle function may be performedby the NLP component. The application selector component can compare thefunctionality of each digital assistant application with the actionspecified by the command.

The method 300 can include selecting the digital assistant application(BLOCK 330). For example, the application selector component can selectthe digital assistant application determined to be able to support theaction specified by the command. When the functionality of the digitalassistant application hosted on the vehicle can support the actionspecified by the command, the application selector component can selectthe digital assistant application hosted on the vehicle, in lieu of theapplication accessible via the client device. When the functionality ofthe digital assistant application accessible via the client device cansupport the action specified by the command, the application selectorcomponent can select the digital assistant application hosted on thevehicle, in lieu of the application hosted on the vehicle. When bothdigital assistant applications can support the action specified by thecommand, the application selector component can use other factors (e.g.,latency or computing resources consumed) to select one of the multipledigital assistant applications.

The method 300 can include invoking the selected digital assistantapplication (BLOCK 335). For instance, the application selectorcomponent can invoke the digital assistant application selected based onthe determination that the application can support the action supportedby the command. The application selector component can call the directaction API of the selected digital assistant application using theaction data structure for the command. The action data structure may bemodified by the application selector component using application datafrom the non-selected digital assistant application.

FIG. 4 is a block diagram of an example computer system 400. Thecomputer system or computing device 400 can include or be used toimplement the system 100 or its components such as the data processingsystem 102. The computing system 400 includes a bus 405 or othercommunication component for communicating information and a processor410 or processing circuit coupled to the bus 405 for processinginformation. The computing system 400 can also include one or moreprocessors 410 or processing circuits coupled to the bus for processinginformation. The computing system 400 also includes main memory 415,such as a random access memory (RAM) or other dynamic storage device,coupled to the bus 405 for storing information and instructions to beexecuted by the processor 410. The main memory 415 can be or include thedata repository 122. The main memory 415 can also be used for storingposition information, temporary variables, or other intermediateinformation during execution of instructions by the processor 410. Thecomputing system 400 may further include a read-only memory (ROM) 420 orother static storage device coupled to the bus 405 for storing staticinformation and instructions for the processor 410. A storage device425, such as a solid state device, magnetic disk or optical disk, can becoupled to the bus 405 to persistently store information andinstructions. The storage device 425 can include or be part of the datarepository 122.

The computing system 400 may be coupled via the bus 405 to a display435, such as a liquid crystal display or active matrix display, fordisplaying information to a user. An input device 430, such as akeyboard including alphanumeric and other keys, may be coupled to thebus 405 for communicating information and command selections to theprocessor 410. The input device 430 can include a touch screen display435. The input device 430 can also include a cursor control, such as amouse, a trackball, or cursor direction keys, for communicatingdirection information and command selections to the processor 410 andfor controlling cursor movement on the display 435. The display 435 canbe part of the data processing system 102, the client computing devices132, or other component of FIG. 1 , for example.

The processes, systems and methods described herein can be implementedby the computing system 400 in response to the processor 410 executingan arrangement of instructions contained in main memory 415. Suchinstructions can be read into main memory 415 from anothercomputer-readable medium, such as the storage device 425. Execution ofthe arrangement of instructions contained in main memory 415 causes thecomputing system 400 to perform the illustrative processes describedherein. One or more processors in a multi-processing arrangement mayalso be employed to execute the instructions contained in main memory415. Hard-wired circuitry can be used in place of or in combination withsoftware instructions together with the systems and methods describedherein. Systems and methods described herein are not limited to anyspecific combination of hardware circuitry and software.

Although an example computing system has been described in FIG. 4 , thesubject matter including the operations described in this specificationcan be implemented in other types of digital electronic circuitry or incomputer software, firmware, or hardware, including the structuresdisclosed in this specification and their structural equivalents, or incombinations of one or more of them.

For situations in which the systems discussed herein collect personalinformation about users, or may make use of personal information, theusers may be provided with an opportunity to control whether programs orfeatures that may collect personal information (e.g., information abouta user's social network, social actions, or activities; a user'spreferences; or a user's location), or to control whether or how toreceive content from a content server or other data processing systemthat may be more relevant to the user. In addition, certain data may beanonymized in one or more ways before it is stored or used, so thatpersonally identifiable information is removed when generatingparameters. For example, a user's identity may be anonymized so that nopersonally identifiable information can be determined for the user, or auser's geographic location may be generalized where location informationis obtained (such as to a city, postal code, or state level), so that aparticular location of a user cannot be determined. Thus, the user mayhave control over how information is collected about him or her and usedby the content server.

The subject matter and the operations described in this specificationcan be implemented in digital electronic circuitry or in computersoftware, firmware, or hardware, including the structures disclosed inthis specification and their structural equivalents, or in combinationsof one or more of them. The subject matter described in thisspecification can be implemented as one or more computer programs, e.g.,one or more circuits of computer program instructions, encoded on one ormore computer storage media for execution by, or to control theoperation of, data processing apparatuses. Alternatively or in addition,the program instructions can be encoded on an artificially generatedpropagated signal, e.g., a machine-generated electrical, optical, orelectromagnetic signal that is generated to encode information fortransmission to suitable receiver apparatus for execution by a dataprocessing apparatus. A computer storage medium can be, or be includedin, a computer-readable storage device, a computer-readable storagesubstrate, a random or serial-access memory array or device, or acombination of one or more of them. While a computer storage medium isnot a propagated signal, a computer storage medium can be a source ordestination of computer program instructions encoded in an artificiallygenerated propagated signal. The computer storage medium can also be, orbe included in, one or more separate components or media (e.g., multipleCDs, disks, or other storage devices). The operations described in thisspecification can be implemented as operations performed by a dataprocessing apparatus on data stored on one or more computer-readablestorage devices or received from other sources.

The terms “data processing system,” “computing device,” “component,” or“data processing apparatus” encompass various apparatuses, devices, andmachines for processing data, including, by way of example, aprogrammable processor, a computer, a system on a chip, or multipleones, or combinations of the foregoing. The apparatus can includespecial-purpose logic circuitry, e.g., an FPGA (field-programmable gatearray) or an ASIC (application-specific integrated circuit). Theapparatus can also include, in addition to hardware, code that createsan execution environment for the computer program in question, e.g.,code that constitutes processor firmware, a protocol stack, a databasemanagement system, an operating system, a cross-platform runtimeenvironment, a virtual machine, or a combination of one or more of them.The apparatus and execution environment can realize various differentcomputing model infrastructures, such as web services, distributedcomputing and grid computing infrastructures. The components of system100 can include or share one or more data processing apparatuses,systems, computing devices, or processors.

A computer program (also known as a program, software, softwareapplication, app, script, or code) can be written in any form ofprogramming language, including compiled or interpreted languages,declarative or procedural languages, and can be deployed in any form,including as a stand-alone program or as a module, component,subroutine, object, or other unit suitable for use in a computingenvironment. A computer program can correspond to a file in a filesystem. A computer program can be stored in a portion of a file thatholds other programs or data (e.g., one or more scripts stored in amarkup language document), in a single file dedicated to the program inquestion, or in multiple coordinated files (e.g., files that store oneor more modules, sub programs, or portions of code). A computer programcan be deployed to be executed on one computer or on multiple computersthat are located at one site or distributed across multiple sites andinterconnected by a communication network.

The processes and logic flows described in this specification can beperformed by one or more programmable processors executing one or morecomputer programs (e.g., components of the data processing system 102)to perform actions by operating on input data and generating output. Theprocesses and logic flows can also be performed by, and apparatuses canalso be implemented as, special purpose logic circuitry, e.g., an FPGA(field-programmable gate array) or an ASIC (application-specificintegrated circuit). Devices suitable for storing computer programinstructions and data include all forms of non-volatile memory, mediaand memory devices, including by way of example semiconductor memorydevices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks,e.g., internal hard disks or removable disks; magneto optical disks; andCD-ROM and DVD-ROM disks. The processor and the memory can besupplemented by, or incorporated in, special purpose logic circuitry.

The subject matter described herein can be implemented in a computingsystem that includes a back end component, e.g., as a data server, orthat includes a middleware component, e.g., an application server, orthat includes a front end component, e.g., a client computer having agraphical user interface or a web browser through which a user caninteract with an implementation of the subject matter described in thisspecification, or a combination of one or more such back end,middleware, or front end components. The components of the system can beinterconnected by any form or medium of digital data communication,e.g., a communication network. Examples of communication networksinclude a local area network (“LAN”) and a wide area network (“WAN”), aninter-network (e.g., the Internet), and peer-to-peer networks (e.g., adhoc peer-to-peer networks).

The computing system such as system 100 or system 400 can includeclients and servers. A client and server are generally remote from eachother and typically interact through a communication network (e.g., thenetwork 146). The relationship of client and server arises by virtue ofcomputer programs running on the respective computers and having aclient-server relationship to each other. In some implementations, aserver transmits data (e.g., data packets representing a content item)to a client device (e.g., for purposes of displaying data to andreceiving user input from a user interacting with the client device).Data generated at the client device (e.g., a result of the userinteraction) can be received from the client device at the server (e.g.,received by the data processing system 102 from the client devices 132or the vehicle 144).

While operations are depicted in the drawings in a particular order,such operations are not required to be performed in the particular ordershown or in sequential order, and all illustrated operations are notrequired to be performed. Actions described herein can be performed in adifferent order.

The separation of various system components does not require separationin all implementations, and the described program components can beincluded in a single hardware or software product. For example, the NLPcomponent 106 and the direct action API 112 can be a single component,app, or program, or a logic device having one or more processingcircuits, or part of one or more servers of the data processing system102.

Having now described some illustrative implementations, it is apparentthat the foregoing is illustrative and not limiting, having beenpresented by way of example. In particular, although many of theexamples presented herein involve specific combinations of method actsor system elements, those acts and those elements may be combined inother ways to accomplish the same objectives. Acts, elements, andfeatures discussed in connection with one implementation are notintended to be excluded from a similar role in other implementations.

The phraseology and terminology used herein is for the purpose ofdescription and should not be regarded as limiting. The use of“including,” “comprising,” “having,” “containing,” “involving,”“characterized by,” “characterized in that,” and variations thereofherein, is meant to encompass the items listed thereafter, equivalentsthereof, and additional items, as well as alternate implementationsconsisting of the items listed thereafter exclusively. In oneimplementation, the systems and methods described herein consist of one,each combination of more than one, or all of the described elements,acts, or components.

Any references to implementations, elements, or acts of the systems andmethods herein referred to in the singular may also embraceimplementations including a plurality of these elements, and anyreferences in plural to any implementation, element, or act herein mayalso embrace implementations including only a single element. Referencesin the singular or plural form are not intended to limit the presentlydisclosed systems or methods, their components, acts, or elements tosingle or plural configurations. References to any act or element beingbased on any information, act, or element may include implementationswhere the act or element is based at least in part on any information,act, or element.

Any implementation disclosed herein may be combined with any otherimplementation or embodiment, and references to “an implementation,”“some implementations,” “one implementation,” or the like are notnecessarily mutually exclusive and are intended to indicate that aparticular feature, structure, or characteristic described in connectionwith the implementation may be included in at least one implementationor embodiment. Such terms as used herein are not necessarily allreferring to the same implementation. Any implementation may be combinedwith any other implementation, inclusively or exclusively, in any mannerconsistent with the aspects and implementations disclosed herein.

References to “or” may be construed as inclusive so that any termsdescribed using “or” may indicate any of a single, more than one, andall of the described terms. A reference to “at least one of ‘A’ and ‘B’”can include only ‘A’, only ‘B’, as well as both ‘A’ and ‘B’. Suchreferences used in conjunction with “comprising” or other openterminology can include additional items.

Where technical features in the drawings, detailed description, or anyclaim are followed by reference signs, the reference signs have beenincluded to increase the intelligibility of the drawings, detaileddescription, and claims. Accordingly, neither the reference signs northeir absence have any limiting effect on the scope of any claimelements.

The systems and methods described herein may be embodied in otherspecific forms without departing from the characteristics thereof. Theforegoing implementations are illustrative rather than limiting of thedescribed systems and methods. Scope of the systems and methodsdescribed herein is thus indicated by the appended claims, rather thanthe foregoing description, and changes that come within the meaning andrange of equivalency of the claims are embraced therein.

What is claimed:
 1. A method implemented by one or more processors of avehicle computing device of a vehicle, the method comprising: receiving,via one or more microphones of the vehicle computing device, an inputaudio signal that captures (i) a given trigger word or phrase; and (ii)a request; identifying, based on processing the input audio signal, agiven digital assistant application, from among a plurality of digitalassistant applications that are accessible by the vehicle computingdevice, that is associated with the given trigger word or phrase;identifying, based on processing the input audio signal, an action to beperformed responsive to the request; and causing the given digitalassistant application to perform the action to satisfy the request. 2.The method of claim 1, wherein the plurality of digital assistantapplications that are accessible by the vehicle computing device includeat least a first digital assistant application and a second digitalassistant application.
 3. The method of claim 2, further comprising: inresponse to determining that the given trigger word or phrase is a firsttrigger word or phrase that is associated with the first digitalassistant application: identifying the first digital assistantapplication as the given digital assistant application; and in responseto determining that the given trigger word or phrase is a second triggerword or phrase that is associated with the second digital assistantapplication: identifying the second digital assistant application as thegiven digital assistant application.
 4. The method of claim 3, whereinthe plurality of digital assistant applications that are accessible bythe vehicle computing device further include a third digital assistantapplication.
 5. The method of claim 4, further comprising: in responseto determining that the given trigger word or phrase is a third triggerword or phrase that is associated with the third digital assistantapplication: identifying the third digital assistant application as thegiven digital assistant application
 6. The method of claim 1, wherein anindication of the given digital assistant application is stored inassociation with the given trigger word or phrase in one or moredatabases that are accessible to the vehicle computing device.
 7. Themethod of claim 6, wherein a corresponding indication of each of theplurality of digital assistant applications is stored in associationwith a corresponding trigger word or phrase in one or more of thedatabases that are accessible to the vehicle computing device.
 8. Themethod of claim 1, wherein identifying the given digital assistantapplication that is associated with the given trigger word or phrase andbased on processing the input audio signal comprises: parsing, using anatural language processor component of the vehicle computing device, afirst portion of the input audio signal to identify the given word orphrase; and determining, based on parsing the input audio signal toidentify the given word or phrase, that the given trigger word or phraseis associated with the given digital assistant application.
 9. Themethod of claim 8, wherein identifying the action to be performedresponsive to the request and based on processing the input audio signalcomprises: parsing, using the natural language processor component ofthe vehicle computing device, a second portion of the input audio signalto identify the request; and determining, based on parsing the inputaudio signal to the request, the action to be performed responsive tothe request.
 10. The method of claim 1, wherein the action to beperformed responsive to the request includes one or more of: changingtemperature controls of the vehicle, actuating one or more locks orwindows of the vehicle, turning one or off a device of the vehicle, orcontrolling playback of media via one or more speakers of the vehiclecomputing device.
 11. A vehicle computing device of a vehicle, thevehicle computing device comprising: one or more hardware processors;and memory storing instructions that, when executed by the one or morehardware processors, cause the one or more hardware processors to:receive, via one or more microphones of the vehicle computing device, aninput audio signal that captures (i) a given trigger word or phrase; and(ii) a request; identify, based on processing the input audio signal, agiven digital assistant application, from among a plurality of digitalassistant applications that are accessible by the vehicle computingdevice, that is associated with the given trigger word or phrase;identify, based on processing the input audio signal, an action to beperformed responsive to the request; and cause the given digitalassistant application to perform the action to satisfy the request. 12.The vehicle computing device of claim 1, wherein the plurality ofdigital assistant applications that are accessible by the vehiclecomputing device include at least a first digital assistant applicationand a second digital assistant application.
 13. The vehicle computingdevice of claim 12, wherein the instructions further cause the one ormore hardware processors to: in response to determining that the giventrigger word or phrase is a first trigger word or phrase that isassociated with the first digital assistant application: identify thefirst digital assistant application as the given digital assistantapplication; and in response to determining that the given trigger wordor phrase is a second trigger word or phrase that is associated with thesecond digital assistant application: identify the second digitalassistant application as the given digital assistant application. 14.The vehicle computing device of claim 13, wherein the plurality ofdigital assistant applications that are accessible by the vehiclecomputing device further include a third digital assistant application.15. The vehicle computing device of claim 14, wherein the instructionsfurther cause the one or more hardware processors to: in response todetermining that the given trigger word or phrase is a third triggerword or phrase that is associated with the third digital assistantapplication: identifying the third digital assistant application as thegiven digital assistant application
 16. The vehicle computing device ofclaim 11, wherein an indication of the given digital assistantapplication is stored in association with the given trigger word orphrase in one or more databases that are accessible to the vehiclecomputing device.
 17. The vehicle computing device of claim 6, wherein acorresponding indication of each of the plurality of digital assistantapplications is stored in association with a corresponding trigger wordor phrase in one or more of the databases that are accessible to thevehicle computing device.
 18. The vehicle computing device of claim 11,wherein the instructions to identify the given digital assistantapplication that is associated with the given trigger word or phrase andbased on processing the input audio signal comprise instructions to:parsing, using a natural language processor component of the vehiclecomputing device, a first portion of the input audio signal to identifythe given word or phrase; and determining, based on parsing the inputaudio signal to identify the given word or phrase, that the giventrigger word or phrase is associated with the given digital assistantapplication.
 19. The vehicle computing device of claim 18, wherein theinstructions to identify the action to be performed responsive to therequest and based on processing the input audio signal compriseinstructions to: parsing, using the natural language processor componentof the vehicle computing device, a second portion of the input audiosignal to identify the request; and determining, based on parsing theinput audio signal to the request, the action to be performed responsiveto the request.
 20. A non-transitory computer-readable storage mediumstoring instructions that, when executed, cause one or more hardwareprocessors of a vehicle computing device of a vehicle to performoperations, the operations comprising: receiving, via one or moremicrophones of the vehicle computing device, an input audio signal thatcaptures (i) a given trigger word or phrase; and (ii) a request;identifying, based on processing the input audio signal, a given digitalassistant application, from among a plurality of digital assistantapplications that are accessible by the vehicle computing device, thatis associated with the given trigger word or phrase; identifying, basedon processing the input audio signal, an action to be performedresponsive to the request; and causing the given digital assistantapplication to perform the action to satisfy the request.